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Analysis of "SIR" 
("Signal"-to-"Interference"-Ratio) in 
Discrete-Time Autonomous Linear Networks 
with Symmetric Weight Matrices 

Zekeriya Uykan 



Abstract 

It's well-known that in a traditional discrete-time autonomous linear systems, the eigenvalues of the 
weigth (system) matrix solely determine the stability of the system. If the spectral radius of the system 
matrix is larger than 1, then the system is unstable. In this paper, we examine the linear systems with 
symmetric weight matrix whose spectral radius is larger than 1 . 

The author introduced a dynamic-system-version of "Signal-to-interference Ratio (SIR)" in non- 
linear networks in [7] and [8] and in continuous-time linear networks in [9]. Using the same "SIR" 
concept, we, in this paper, analyse the "SIR" of the states in the following two iV-dimensional discrete- 
time autonomous linear systems: 1) The system x(fc + 1) = (I + a(— rl + W))x(fc) which is obtained 
by discretizing the autonomous continuous-time linear system in [9] using Euler method; where I is the 
identity matrix, r is a positive real number, and a > is the step size. 2) A more general autonomous 
linear system descibed by x(fc + 1) = — pi + Wx(fc), where W is any real symmetric matrix whose 
diagonal elements are zero, and I denotes the identity matrix and p is a positive real number. Our 
analysis shows that: 1) The "SIR" of any state converges to a constant value, called "Ultimate SIR", 
in a finite time in the above-mentioned discrete-time linear systems. 2) The "Ultimate SIR" in the 
first system above is equal to — where X max is the maximum (positive) eigenvalue of the matrix 
W. These results are in line with those of [9] where corresponding continuous-time linear system is 
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examined. 3) The "Ultimate SIR" in the second system above is equal to where A m is the eigenvalue 
of W which satisfy |A m — p\ = max{|Ai — if p is accordingly determined from the interval 

< p< 1. 

In the later part of the paper, we use the introduced "Ultimate SIR" to stabilize the (originally unsta- 
ble) networks. It's shown that the proposed Discrete-Time "Stabilized"-Autonomous-Linear-Networks- 
with-Ultimate-SIR" exhibit features which are generally attributed to Discrete-Time Hopfield Networks. 
Taking the sign of the converged states, the proposed networks are applied to binary associative memory 
design. Computer simulations show the effectiveness of the proposed networks as compared to traditional 
discrete Hopfield Networks. 



Index Terms 

Autonomous Discrete-Time Linear Systems, discrete Hopfield Networks, associative memory sys- 
tems, Signal to Interference Ratio (SIR). 



I. INTRODUCTION 

Signal-to-Interference Ratio (SIR) is an important entity in commucations engineering which 
indicates the quality of a link between a transmitter and a receiver in a multi transmitter-receiver 
environment (see e.g. [4], among many others). For example, let N represent the number of 
transmitters and receivers using the same channel. Then the received SIR at receiver % is given 
by (see e.g. [4]) 



g/fi i (fc)= 7 i(fc) = .. , JT W _„,„ ^ = l,...,N (1) 



9aPi{k) 

where Pi(k) is the transmission power of transmitter % at time step k, gij is the link gain 
from transmitter j to receiver i (e.g. in case of wireless communications, involves path 
loss, shadowing, etc) and Vi represents the receiver noise at receiver i. Typically, in wireless 
communication systems like cellular radio systems, every transmitter tries to optimize its power 
Pi(k) such that the received SIR(k) (i.e., ^(k)) in eq.(l) is kept at a target SIR value, 7* 5 *. In 
an interference dominant scenario, the receiver background noise z/j can be ignored and then 

= yST T^TTy i = l,. -.,N (2) 
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The author defines the following dynamic-system-version of "Signal-to-Interference-Ratio 
(SIR)", denoted by 9i(k), by rewriting the eq.(l) with neural network terminology in [7] and 
[8]: 



where 0j(A;) is the defined ficticious "SIR" at time step k, Xi(k) is the state of the i'th neuron, 
an is the feedback coefficient from its state to its input layer, Wij is the weight from the output 
of the j'th neuron to the input of the j'th neuron. For the sake of brevity, in this paper, we 
assume the "interference dominant" case, i.e. z/; is negligible. 

A traditional discrete-time autonomous linear network is given by 

x(ib + 1) = Mx(jfe), x(ife) G R Nxl , M G R NxN , (4) 

where x(fc) shows the state vector at time t and square matrix M is called system matrix or 
weight matrix. 

It's well-known that in the system of eq.(4), the eigenvalues of the weight (system) matrix 
solely determine the stability of system. If the spectral radius of the matrix is larger than 1, then 
the system is unstable. In this paper, we examine the linear systems with system matrices whose 
spectral radius is larger than 1 . 

Using the same "SIR" concept in eq.(3), we, in this paper, analyse the "SIR" of the states in 
the following two discrete-time autonomous linear systems: 

1) The following linear system which is obtained by discretizing the continuous-time linear 
system x = (-rl + W)x in [9] using Euler method: 

jc(k + 1) = (i + a(-rl + W))x(ib) (5) 

where I denotes the identity matrix, (— rl + W) is the real symmetric system matrix with 
zero-diagonal W, and a is the step size. 

2) A more general autonomous linear system descibed by 

x(Jfe + 1) = (-rl + W)x(Jfe) (6) 
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where W is any real symmetric matrix whose diagonal elements are zero, and I denotes 
the identity matrix and r is a positive real number. 

Our analysis shows that: 1) The "SIR" of any state converges to a constant value, called 
"Ultimate SIR", in a finite time in the above-mentioned discrete-time linear systems. 2) The 
"Ultimate SIR" in the first system above is equal to -r- 2 — where \ max is the maximum (positive) 
eigenvalue of the matrix W. These results are in line with those of [9] where corresponding 
continuous-time linear system is examined. 3) The "Ultimate SIR" in the second system above 
is equal to where A m is the eigenvalue of W which satisfy |A m — p\ = max{|Aj — p\}f =1 if 
p is accordingly determined from the interval < p < 1. 

In the later part of the paper, we use the introduced "Ultimate SIR" to stabilize the (originally 
unstable) network. It's shown that the proposed Discrete-Time "Stabilized"-Autonomous-Linear- 
Networks-with-Ultimate-SIR" exhibit features which are generally attributed to Discrete-Time 
Hopfield Networks. Taking the sign of the converged states, the proposed networks are applied 
to binary associative memory design. 

The paper is organized as follows: The ultimate "SIR" is analysed for the autonomous linear 
discrete-time systems with symmetric weight matrices in section II. Section III presents the 
stabilized networks by their Ultimate "SIR" to be used as a binary associative memory system. 
Simulation results are presented in section IV, which is followed by the conclusions in Section 
V. 

II. Analysis of "SIR" in Discrete-Time Autonomous Linear Networks with 

Symmetric Weight Matrices 

In this section, we analyse the "SIR" of the states in the following two discrete-time au- 
tonomous linear systems: 1) The discrete-time autonomous system which is obtained by dis- 
cretizing the continuous-time linear system in [9] using Euler method; and 2) A more general 
autonomous linear system descibed by x(/c + 1) = (—pi + W)x(fc), where W is any real 
symmetric matrix whose diagonal elements are zero, and I denotes the identity matrix and r is 
a positive real number. 

A. Discretized Autonomous Linear Systems with symmetric matrix case 

The author examines the "SIR" in the following continuous-time linear system in [9]: 
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x = ( - rl + W)x 



(7) 



where x shows the derivative of x with respect to time, i.e., x = ^. In this subsection, we 
analyse the following discrete-time autonomous linear system which is obtained by discretizing 
the continuous-time system of eq.(7) by using well-known Euler method: 



x(Jfe + 1) = (I + a(-rl + W))x(ife) (8) 

where I is the identity matrix, r is a positive real number, (— rl + W) is the system matrix, 
x(fc) shows the state vector at step k, and a > is the step size and 



rl = 



r 
r 









r 



W 



NxN 





W 2 1 



Wl2 




Wm w N2 



Wi N 




(9) 



NxN 



In this paper, we examine only the linear systems with symmetrix weight matrices, i.e., Wij = 



w 



1,2, ... ,N. It's well known that desigining the weight matrix W as a symmetric 



one yields that all eigenvalues are real (see e.g. [6]), which we assume throughout the paper due 
to the simplicity and brevity of its analysis. 

The reason of the notation in (9) is because we prefer to have the same notation as in [7]. 

Proposition 1: 

In the autonomous discrete-time linear network of eq.(8), let's assume that the spectral radius 
of the system matrix (I + a(— rl + W)) is larger than 1. (This assumption is equal to the 
assumption that W has positive eigenvalue(s) and r > is chosen such that \ max > r, where 
Knax is the maximum (positive) eigenvalue of W). If a is chosen such that < ar < 1, then the 
defined "SIR" 9i(k) in eq.(3) for any state % converges to the following constant within a finite 
step number for any initial vector x(0) which is not completely perpendicular to the eigenvector 
corresponding to the largest eigenvalue of W. 1 

1 It's easy to check in advance if the initial vector x(0) is completely perpendicular to the eigenvector of the maximum 
(positive) eigenvalue of W or not. If this is the case, then this can easily be overcome by introducing a small random variable 
to x(0) so that it's not completely perpendicular to the mentioned eigenvector. 



March 19, 2009 



DRAFT 



105 



6i(k > kr) = t— — , i = l,2,...,iV (10) 

An 



*max 



where \ max is the maximum (positive) eigenvalue of the weight matrix W and hp shows a 
finite time constant. 
Proof: 

From eq. (8), it's obtained 

x(ife) = (i + a(-rl + W)) fc x(0) (11) 

where x(0) shows the initial state vector at step zero. Let us first examine the powers of the 
matrix (l + a(-rI + W)) in (11) in terms of matrix rl and the eigenvectors of matrix W: 
It's well known that any symmetric real square matrix can be decomposed into 

N N 

W = $>v iV f = 5>V, (12) 

i=l i=l 

where {XijfLi an d {vi}f =1 show the (real) eigenvalues and the corresponding eigenvectors 
and the eigenvectors {v;}^ are orthonormal (see e.g. [6]), i.e., 

1 if i = j, where i,j — l,2,...,N 
ViVj = < (13) 
if i^j, 

Let's define the outer-product matrices of the eigenvectors {\i}f =1 as V 3 - = Vjvf , i = 
1, 2, . . . , N, which, from eq.(13), is equal to 

I if % — j, where i. j — 1, 2, . . . , N 

v = <{ ^ ,j , , , (14) 

if ^j, 
where I is the identity matrix. Defining matrix M, 

M = I + a(—rl + W) (15) 

which is obtained as 

N 

M = (l-or)I + X;A(l)Vi (16) 

i=l 
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where r > 0, a > 0, and where $(1) is equal to 

= a\ (17) 

The matrix M 2 can be written as 

N 

M 2 = (l-ar) 2 I + ]TA(2)V, ( 1 8) 

i=i 



where $ (2) is equal to 

A(2) = a(l - ar)Ai + (1 - ar + aA^l) (19) 
Similarly, the matrix M 3 can be written as 



M 3 = (l-mfl + £A(3)V, (20) 

where /3j(3) is equal to 



E 

i=i 



A(3) = a(l - mfA, + (1 - ar + aA,)A(2) (21) 
So, M 4 can be written as 

N 

M 4 = (l-ar) 4 I + ]TA(4)V, (22) 
i=i 

where A (4) is equal to 

A(4) = a(l - arf\ + (1 - ar + aA,)A(3) (23) 
So, at step fc, the matrix (M) fc is obtained as 

N 

M k = (1 - mfl + ]T A(^)V, (24) 
i=i 

where Pi(k) is equal to 

Pi(k) = a(l - ar) k ~ l \i + (1 + a{\ - r))p t (k - 1) (25) 
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Using eq.(17) and (25), the j3i(k) is obtained as 



A(l) = a\ (26) 

ft(2) = a\i((l - ar) + (1 + a(Xi - r))) (27) 

A(3) = aX^l-arf + (l-ar)(l + a(X i -r)) + (l+a(X l -r)f) (28) 

i (29) 

A; 

Pi(k) = aA l ^(l-ar) fc - m (l + a(A,-r)r- 1 (30) 

m=l 

Defining X { = Q(l — ctr), we obtain 

(1 - ar) k - m (l + a(X t - r)) m ~ l = (1 - ^(l + aQ) m ~ l (31) 
Writing eq.(31) in eq.(30) gives 

Pi(k) = a&(l - ar) k S(k) (32) 

where S(k) is 

k 

S(k)= E(l + «CO m_1 (33) 

m=l 

Summing —S(k) with (1 + aQ)S(k) yields 

= d + ^'-i (34) 

From eq.(32), (33) and (34), we obtain 

= (1 - ur)\l + «Ci)* " (1 " oirf (35) 
Using the definition Q = Aj/(1 — or) in eq.(35) gives 

A(^) = (1 + «(Ai - r)) fe - (1 - or)* (36) 

From eq.(24) and eq.(36), 
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N N 

M k = (1 - arfl + E (1 + - r)) fe V, - £(1 - arfV, (37) 
i=i i=i 

Let's put the N eigenvalues of matrix W into two groups as follows: Let those eigenvalues 
which are smaller that r, belong to set T = {A^j^Li where N t is the length of the set; and let 
those eigenvalues which are larger than r belong to set S = {Xj s }f s s =1 where N s is the length 
of the set. We write the matrix M fe in eq.(37) using this eigenvalue grouping 

M k = M tp (k) + M sp (k) (38) 

where 

N 

M tp (k) = (1 - arfl - ^(1 - arfY, + ^ (1 + a(X jt - r)) k V 3t (39) 

»=i iter 

and 

M sp (k)=Y,(l + *(K-r)) k V 3s (40) 

jsGS 

We call the matrices ~M tp (k) and M sp (A;) in (39) and (40) as transitory phase part and steady 
phase part, respectively, of the matrix M fc . 

It's observed from eq.(39) that the M_ tp (k) converges to zero in a finite step number k T because 
relatively small step number a > is chosen such that (1 — ar) < 1 and 1 +a(Xj t — r) < 1. So, 

Mtp(fc) ^0, k > k T (41) 

Thus, what shapes the steady state behavior of the system in eq.(ll) and (15) is merely the 
M. sp (k) in eq.(40 ). So, the steady phase solution is obtained from eq.(ll), (15) and (40) using 
the above observations as follows 

x sp (k) = M sp (A;)x(0) (42) 
= ]T (l + a(A, s -r)) fe V, s x(0), k > k T (43) 

Let's define the interference vector, 3 sp (k) as 
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Js P (k) = W Xsp (fc) (44) 
Using eq.(12) in (44) and the orthonormal features in (14) yields 

J sp (k) = £ K0- + a (K - O) fc V Js x(0) (45) 

JsGS 

First defining V 3 -x(0) = u,, and £ = jf^, then dividing vector x sp (fc) of eq.(43) to J sp (A;) of 
eq.(45) elementwise and comparing the outcome with the "SIR" definition in eq.(3) results in 



XspAk) - -9i(k), i = l,...,N (46) 

r 



Jsp,i\k) 



E JseS (i + ^)S aiI (47) 



In eq.(47), we assume that the Uj = Vjx(O) which corresponds to the eigenvector of the largest 
positive eigenvalue is different than zero vector. This means that we assume in the analysis here 
that x(0) is not completely perpendicular to the mentioned eigenvector. This is something easy 
to check in advance. If it is the case, then this can easily be overcome by introducing a small 
random number to x(0) so that it's not completely perpendicular to the mentioned eigenvector. 

From the analysis above, we observe that 

1) If all the (positive) eigenvalues greater than r are the same, which is denoted as X b , then 
it's seen from (47) that 



6i{k) = —, i = l,...,N, k>k T (48) 
At 

2) Similarly, if there is only one positive eigenvalue which is larger than r, shown as X b , then 
eq.(48) holds. 

3) If there are more than two different (positive) eigenvalues and the largest positive eigen- 
value is single (not multiple), then we see from (46) that the term related to the largest 
(positive) eigenvalue dominates the sum of the nominator. Same observation is valid for the 
sum of the denominator. This is because a relatively small increase in \j causes exponential 
increase as time step evolves, which is shown in the following: Let's show the two largest 
(positive) eigenvalues as X max and Xj respectively and the difference between them as AA. 

March 19, 2009 DRAFT 



110 



So, \ m ax = Aj + AA. Let's define the following ratio between the terms related to the two 
highest eigenvalues in the nominator 

*"<*> = (1 + Va/+ AA))* <49) 

where 



(50) 



1 — ar 

Similarly, let's define the ratio between the terms related to the two highest eigenvalues 
in the denominator as 

From eq.(49) and (51), since < 1 due to the above assumptions, 

K d (k) < K n (k) (52) 

We plot the ratio K n (k) in Fig. 1 for some different AA values and for a typical £ value. 
The Figure 1 and eq.(52) implies that the terms related to the \ rnax dominate the sum of 
the nominator and that of the denominator respectively. So, from eq.(47) and (50), 



x 



i(k) _ V,„ s ( 1 + i\ js ) k u^ (1 + i\ max f 



Sp,l\-~ I ^J 3 kzO \~ ' -Js/ -Js,t ^ i ^■■miiij - ^ ^ (53) 

■Jsp,i(k) Sji s gS Ajr s (1 + £Aj s ) fc- Uj 3) j A maa; (l + £,\ m ax) k ^max 

4) If the largest positive eigenvalue is a multiple eigenvale, then, similarly, the corresponding 
terms in the sum of the nominator and that of the demoninator become dominant, which 
implies from eq.(47), (49) and (51) that x / p ' i( ^ converges to as step number increases. 

Using the observations 1 to 4, eq.(41), the "SIR" definition in eq.(3), eq.(46) and (47), we 
conclude that 

»<(*) = — N TX "fl = k>k T i = l,...,N, (54) 

where \ max is the largest (positive) eigenvalue of the matrix W, and k T shows the finite time 
constant (during which the matrix M_ tp (k) in eq.(39) vanishes), which completes the proof. 
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Definition: Ultimate SIR value: In proposition 1, we showed that the SIR in (3) for every state 
in the autonomous discrete-time linear networks in eq.(8) converges to a constant value as step 
number goes to infinity. We call this converged constant value as "ultimate SIR" and denote as 

Quit 

B. A more general autonomous linear discrete-time systems with symmetric matrix case 
In this subsection, we analyse the following discrete-time autonomous linear system 

x(Jfe + 1) = (-pi + W)x(ife) (55) 

where I is the identity matrix, p is a positive real number, and (—pi + W) is the symmetric 
system matrix. The real symmetric matrix W is shown in eq.(9). 

Proposition 2: 

In the the discrete-time linear system of eq.(55), let's assume that the spectral radius of 
symmetric matrix W in (9) is larger than 1, i.e., the maximum of the norms of the eigenvalues 
is larger than 1. 

If p is chosen such that 

1) 

< p < 1, (56) 

and 

2) Define the eigenvalue(s) \ m as 

\\ m -p\=max{\X i -p\}g 1 > 1 (57) 

the eigenvalue A m is unique. (In other words, p is chosen in such a way that the equation 
eq.(57) does not hold for two eigenvalues with opposite signs. It would hold for a multiple 
eigenvalue as well, i.e., same sign.) 
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then the defined "SIR" (6i(k)) in eq.(3) for any state % converges to the following ultimate 
SIR as step number k evolves for any initial vector x(0) which is not completely perpendicular 
2 to the eigenvector corresponding to the eigenvalue A m in (57) of W. 

9i(k > kr) = y~, i = l,2,...,N (58) 

Am 

where A m is the eigenvalue of the weight matrix W which satisfy eq.(57) and k T shows a 
finite time constant. 
Proof: 
From eq.(55), 

x(Jfe) = (-pi + W) fe x(0) (59) 

where x(0) shows the initial state vector at step zero. Let's examine the powers of (— pI + W) 
in (59) in terms of the eigenvectors of W using eqs.(12)-(14): 

N 

(_ p I + W ) = -pI + 5Xl)V, (60) 

i=l 

where ^(1) is equal to 

r]i{l) = \i (61) 
The matrix (—pi + W) 2 can be written as 

N 



;-pI + W) 2 = p 2 I + ]>X2)V, (62) 



1=1 



where ^(2) is equal to 



^i(2) = —p\ + (Aj - p)r)i(l) (63) 
Similarly, the matrix (—pi + W) 3 can be written as 

2 It's easy to check in advance if the initial vector x(0) is completely perpendicular to the eigenvector of the eigenvalue A m 
in (57) of W or not. If this is the case, then this can easily be overcome by introducing a small random number to x(0) so 
that it's not completely perpendicular to the mentioned eigenvector. 
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N 

-pi + W) 3 = -p 3 I + £ ^(3)Vi (64) 
i=i 



where 7^(3) is equal to 



77i(3) =p 2 A i + (A i -p)r7 i (2) (65) 
So, at step fc, the matrix (—pi + W) fc is obtained as 

N 

(_ p l + W ) fe = (-p) fe I + X: Vi(k)Vi (66) 

i=l 

where ^(fc) is 

Vi (k) = {-pf^Xi + (A, - p)r)i(k - 1) (67) 

So, from (61)-(67) 



^(1) = A, (68) 

^(2) = A i (-p+(A i -p)) (69) 

^(3) = A,(p 2 -p(A t -p) + (A,-p) 2 ) (70) 

i (VI) 

= A i 2(-l) fc "V" m (Ai-p) m - 1 (72) 

m=l 

Defining Aj = p«p, we obtain 

pfe - m(A . _ p)m -i = pfc -i (Ai . _ 1)m -i (73) 
Writing eq.(73) in eq.(72) gives 

rn(k) = Xip'-'Sik) (74) 

where S(k) is 

5(fc)= ^(-l^/^-ir- 1 (75) 

m=l 
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Summing S(k) with (/ij — l)S(k) yields 



m = ( -o~ + (*-!)» (76) 

(J>i 



From eq.(74), (75) and (76), we obtain 

„ W = V- (- 1 ' t - 1 + C-- 1 ' t (77) 
Using the definition /j; = Aj/p in eq.(77) gives 

m {k) = {-l) k - l p k + {\ l -p) k (78) 

From eq.(66) and eq.(78), 

N 

+ W ) fc = (-p) fe I + ]T(-l) fe -y V, + (Aj - p) fe V, (79) 
i=i 

Let's put the N eigenvalues of matrix W into two groups as follows: Let those eigenvalues 
which satisfy \Xj — p\ < 1 belong to set T = {\j t }%Li where N t is the length of the set; and let 
all other eigenvalues (i.e. those which satisfy |Aj — p\ > 1) belong to set S = {\j s }f a s =1 where 
N s is the length of the set. Here, p is chosen such that no eigenvalue satisfy |Aj — p\ = 1. We 
write the matrix (—pi + W) fc in eq.(79) using this eigenvalue grouping as follows 

(_ p l + W ) fe = N tp (k) + N sp (k) (80) 

where 

N tp (k) = (-p) k i + f:(-i) fe -v fc v, + x: (a* - P ) k v Jt (8D 

i=l it ST 

and 

N sp (A;)= E(^.-P) fcv i. (82) 

In (81), |Aj t — p\ < 1 from the grouping mentioned above and p is chosen such that < p < 1, 
which means that the N tp (k) converges to zero in a finite step number k T , i.e., 
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N tp (k) ^ 0, k>k T (83) 

Thus, what shapes the steady state behavior of the system in eq.(59) is merely the 'N sp (k) in 
eq.(82 ). We call the matrices N tp (k) and 'N sp (k) in (81) and (82) as transitory phase part and 
steady phase part, respectively, of the matrix N fc . 

So, the steady phase solution is obtained from eq.(59), (80) (81) and (82) as follows 



x sp (k) = N sp (A;)x(0) (84) 
= E(^-P) fcv ^(0), k>k T (85) 

js&S 

Let's define the interference vector, 3 sp (k) as 

3 sp (k) = Wx sp (fc) (86) 
Using eq.(12) in (86) and the orthonormal features in (14) yields 

J^^^i^-^xfO) (87) 

Defining Vjx(O) = Uj, and then dividing vector x sp (A;) of eq.(85) to 3 sp (k) of eq.(87) 
elementwise and comparing the outcome with the "SIR" definition in eq.(3) results in 



^4tv = - 6 M> i = l,...,N (88) 

_ T,j 3 ts(^js - P) ku js,i 



Ej s esA is (A js -p) k u js ,i 

In eq.(89), we assume that the Uj = Vjx(0) corresponding to the eigenvector A m in eq.(57) 
is different than zero vector. This means that we assume in the analysis here that x(0) is not 
completely perpendicular to the mentioned eigenvector. This is something easy to check in 
advance. If it is the case, then this can easily be overcome by introducing a small random 
number to x(0) so that it's not completely perpendicular to the mentioned eigenvector. 

Here it's assumed that the eigenvalue X m satisfying the equation eq.(57) is unique. In other 
words, p is chosen in such a way that the equation eq.(57) does not hold for two eigenvalues with 
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opposite signs. (It holds for a multiple eigenvalue, i.e., with same sign). Using this assumption 
(which can easily be met by choosing p accoridingly) in eq.(89) yields the following: The term 
related to eigenvalue A m in eq.(57) dominates the sum of the nominator. This is because a 
relatively small decrease in \j causes exponential decrease as time step evolves, which is shown 
in the following: Let's define the following ratio 

where AA represents the decrease. Similarly, for denominator 

Kd(k) = (a~ + aakaI +1a - pj (91) 

From eq.(90) and (91), since j^f-± < 1, 

K d (k) < K n (k) (92) 

We plot some typical examples of the ratio n n (k) in Fig. 2 for some different A A values. The 
Figure 2 and eq.(92) implies that the terms related to the A„ t dominate the sum of the nominator 
and that of the denominator respectively. So, from eq.(89) 



x 



i(k) _ E js es(K - P)%s,i (A m - p) k u jm:i 



sp,t\-~; _ ^ijs^o \- -js r i ~js,<. ^ v -in rj -jm,' _ - k ~> (93) 

Jsp,i(k) Ej s GS ^jsi^js ~ P) ku js,i ^m(A m — p) k Uj m ,i ^max 

where A m is defined by eq.(57). If the largest positive eigenvalue is a multiple eigenvale, 
then, similarly, the corresponding terms in the sum of the nominator and that of the demoninator 
become dominant, which implies from eq.(89), (90)-(93) that Xap ^ k \ converges to -r-^— as step 
number evolves. 

From eq.(83), and the "SIR" definition in eq.(3), we conclude from eq.(88)-(93) that 

Oi{k) = ^ N rXspM — = k>k T < = 1,...,JV, (94) 

where A m is defined by eq.(57), and k T shows the finite time constant (during which the 
matrix N tp (A;) in eq.(81) vanishes), which completes the proof. 



March 19, 2009 



DRAFT 



117 



III. Stabilized Discrete-Time Autonomous Linear Networks with Ultimate 

"SIR" 

The proposed autonomous networks networks are 
1) 

x(fc + l) = (l + «(-rI + W))x(fc)5(0(fc) -6 ult ) (95) 
y(fc) = sign(x(k)) (96) 

where W is defined in (9) a is step size, I is identity matrix and r > as in eq.(8), 
9(k) = [<9i(fc)(9 2 (A;) . . . 6 N (k)] T , and 6 ult = [Of Of . . . 6f] T , and y(Jfe) is the output of the 
network. In eq.(95) 

f if and only if 9(k) = 6 ult , 
5(6 - e ult ) = \ y y i (97) 

I 1 otherwise 

We call the network in ref.(95) as Discrete Stabilized Autonomous Linear Networks by 
Ultimate "SIR"1 (DSAL-U"SIR"1). 

2) 

x(ifc + l) = (-pi + W)x(fc)5(0(fc) - 6 ult ) (98) 
y(k) = sign(x(k)) (99) 

where I is the identity matrix, 1 > p > and W is defined in eq.(9), and y(k) is the 
output of the network. We call the network in ref.(98) as DSAL-U"SIR"2. 

Proposition 3: 

The proposed discrete-time networks of DSAL-U"SIR"1 in eq.(95) is stable for any initial 
vector x(0) which is not completely perpendicular to the eigenvector corresponding to the largest 
eigenvalue of W. 3 

Proof: The proof of proposition 1 above shows that in the linear networks of eq.(8), the 
defined SIR in eq.(3) for state i converges to the constant ultimate SIR value in eq.(10) for any 
initial condition x(0) within a finite step number k T . It's seen that the DSAL-U"SIR"1 in eq.(8) 

3 See the comments of proposition 1. 
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is nothing but the underlying network of the proposed networks SAL-U"SIR"1 without the 5 
function. Since the "SIR" in eq.(3) exponentially approaches to the constant Ultimate "SIR" in 
eq.(10), the delta function eq.(97) will stop the exponential increase once 9(k) = 9 ult , at which 
the system outputs reach their steady state responses. So, the DSAL-U"SIR"1 is stable. 

■ 

Proposition 4: 

The proposed discrete-time network DSAL-U"SIR"2 in (98) is stable for any initial vector 
x(0) which is not completely perpendicular to the eigenvector corresponding to the eigenvalue 
described in eq.(57). 4 

Proof: The proof of proposition 2 above shows that in the linear network of (55), the 
defined SIR in eq.(3) for state i converges to the constant ultimate SIR value in eq. (58), for any 
initial condition x(0) within a finite step number k T . It's seen that the linear networks of eq.(55) 
is nothing but the underlying network of the proposed network DSAL-U"SIR"2 without the 8 
function. Since the "SIR" in eq.(3) exponentially approaches to the constant Ultimate "SIR" in 
eq.(58), the delta function eq.(97) will stop the exponential increase once 9(k) = 9 ult , at which 
the system output reach its steady state response. So, the DSAL-U"SIR"2 is stable. 

■ 

So, from the analysis above for symmetric W and < r < X rnax for the SAL-U"SIR"1 in 
eq.(95) and < p < 1 for the DSAL-U"SIR"2 in eq.(98), we conclude that 

1) The DSALU-"SIR"1 and DSALU-"SIR"2 does not show oscilatory behaviour because it's 
assured by the design parameter r that p, respectively, that there is no eigenvalue on the 
unit circle. 

2) The transition phase of the "unstable" linear network is shaped by the initial state vector 
and the phase space characteristics formed by the eigenvectors of W. The network is 
stabilized by a switch function once the network has passed the transition phase. The 
output of the network then is formed taking the sign of the converged states. If the state 
converges to a plus or minus value is dictated by the phase space of the underlying linear 
network from the initial state vector at time 0. 

4 See the comments of proposition 2. 
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Choosing the r and p in the DSALU-"SIR"1 and DSALU-"SIR"2 respectively such that the 
overall system matrix has positive eigenvalues makes the proposed networks exhibit similar 
features as Hopfield Network does as shown in the simulation results in section IV. 

As far as the design of weight matrix rl (pi) and W is concerned, well-known Hebb-learning 
rule ([5]) is one of the commonly used methods (see e.g. [2]). We proposed a method in [7] 
which is based on the Hebb-learning rule [5]. We summarize the design method here as well 
for the sake of completeness. 



A. Outer products based network design 

Let's assume that L desired prototype vectors, {d s }f =1 , are given from (— 1, +1)^. The 
proposed method is based on well-known Hebb-learning [5] as follows: 

Step 1: Calculate the sum of outer products of the prototype vectors (Hebb Rule, [5]) 

Q = ]TcUj (100) 

s=l 

Step 2: Determine the diagonal matrix rl and W as follows: 

r = qu + $ (101) 

where $ is a real number and 

if i = j, 

i,j = l,...,N (102) 

Qij if i ^ j 



Wij 



where shows the entries of matrix Q, N is the dimension of the vector x and L is the 
number of the prototype vectors (N > L > 0). From eq.(100), qu = L since {d s } is from 

(-!,+!)". 

We assume that the desired prototype vectors are orthogonal and we use the following design 
procedure for matrices A, W and b, which is based on Hebb learning ([5]). 

Proposition 5: 

For the proposed network DSALU-"SIR"1 in eq.(8) whose weight matrix is designed by the 
proposed outer-products (Hebbian-learning)-based method above, if the prototype vectors are 
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orthogonal, then the defined SIR in eq.(3) for any state converges to the following constant 
"ultimate SIR" awithin a finite step number 

0i(k > kr) = j^-j- (103) 

where N is the dimension of the network and L is the prototype vectors, t T shows a finite 
step number for any initial condition x(0) which is not completely orthogonal to any of the raws 
of matrix Q in eq.(100). 5 
Proof: 

The proof is presented in Appendix I. 

■ 

Corollary 1: 

For the proposed DSALU-"SIR"1 to be used as an associate memory system, whose weight 
matrix is designed by the proposed outer-products (Hebbian-learning)-based method in section 
III-A for L orthogonal binary vectors of dimension N, 



X max = N-L (104) 
where \ rnax is the maximum (positive) eigenvalue of the weight matrix W. 
Proof: 

From the proposition 1 and 4 above, the result in proposition 1 is valid for any real symmetric 
matrix W whose maximum eigenvalue is positive, while the result of proposition 4 is for only 
the symmetric matrix designed by the method in section III-A. So, comparing the results of the 
proposition 1 and 4, we conclude that for the network in proposition 4, the maximum (positive) 
eigenvalue of the weight matrix W is equal to N — L. 

U 

5 It's easy to check in advance if the initial vector x(0) is completely orthogonal to any of the raws of matrix Q in eq.(100) or 
not. If so, then this can easily be overcome by introducing a small random number to x(0) so that it's not completely orthogonal 
to any of the raws of matrix Q. 
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IV. Simulation Results 

We take similar examples as in [7], [8] and [9] for the sake of brevity and easy reproduction of 
the simulation results. We apply the same Hebb-based (outer-products-based) design procedure 
([5]) in [7] and [9], which is presented in section III-A. So, the weight matrix W in all the 
simulated networks (the proposed networks and Discrete-Time Hopfield Network) are the same. 

In this section, we present two examples, one with 8 neurons and one with 16 neurons. As 
in [8], traditional Hopfield network is used a reference network. The discrete Hopfield Network 
[1] is 



x fc+1 = sign(Wx k ^ 



(105) 



where W is the weight matrix and x fc is the state at time k, and at most one state is updated. 
Example 1: 

The desired prototype vectors are 



D = 



(106) 



111 1-1-1-1-1 
11-1-11 1-1-1 
The weight matrices rl and W, and the threshold vector b are obtained as follows by using 
the outer-products-based design mentioned above and d is chosen as -1 and for the DSALU- 
U"SIR"2 network, p = 0.5. 



A = 21, W = 
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v = 



(107) 



where I shows the identity matrix of dimension N by N. 
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The Figure 3 shows the percentages of correctly recovered desired patterns for all possible 
initial conditions x(/c = 0) G (— 1,+1) N , in the proposed DSALU-"SIR"1 and 2 as compared 
to traditional Hopfield network. 

Let m d show the number of prototype vectors and C(N, K) represents the combination N, K 
(such that N > K > 0), which is equal to C(N,K) = ( N J^y K , , where ! shows factorial. In 
our simulation, the prototype vectors are from (—1,1)^ as seen above. For initial conditions, 
we alter the sign of K states where K=0, 1, 2, 3 and 4, which means the initial condition is 
within i^-Hamming distance from the corresponding prototype vector. So, the total number of 
different possible combinations for the initial conditions for this example is 24, 84 and 168 for 
1, 2 and 3-Hamming distance cases respectively, which could be calculated by m d x C(8,K), 
where m d = 3 and K — 1, 2 and 3. 

As seen from Figure 3, the performance of the proposed networks DSALU-"SIR"1 and 2 are 
the same as that of the discrete-time Hopfield Network for 1-Hamming distance case (%100 for 
both networks) and are comparable results for 2 and 3-Hamming distance cases respectively. 

Example 2: 

The desired prototype vectors are 

111 1 1 1 1 1-1-1-1-1-1-1-1-1 
D = 111 1-1-1-1-11 1 1 1-1-1-1-1 (108) 

11-1-11 1-1-11 1-1-11 1-1-1 
The weight matrices rl and W and threshold vector b is obtained as follows by using the 
outer products based design as explained above. For matrix rl, d is chosen as -2. The other 
network paramaters are chosen as in example 1. 

rl = 31, 
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(109) 



The Figure 4 shows the percentages of correctly recovered desired patterns for all possible 
initial conditions x(& = 0) G (—1, +1) 16 , in the proposed DSALU"SIR"1 and 2 as compared to 
the traditional Hopfield network. 

The total number of different possible combinations for the initial conditions for this example 
is 64, 480 and 2240 and 7280 for 1, 2, 3 and 4-Hamming distance cases respectively, which 
could be calculated by m d x C(16, K), where m d = 4 and K — 1, 2, 3 and 4. 

As seen from Figure 4 the performance of the proposed networks DSALU-"SIR"1 and 2 are 
the same as that of Hopfield Network for 1 and 2-Hamming distance cases (%100 for both 
networks), and are comparable for 3,4 and 5-Hamming distance cases respectively. 



V. Conclusions 

Using the same "SIR" concept as in [7], and [8], we, in this paper, analyse the "SIR" of the 
states in the following two iV-dimensional discrete-time autonomous linear systems: 
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1) The system x(/c + 1) = (I + a(— rl + W))x(/c) which is obtained by discretizing the 
autonomous continuous-time linear system in [9] using Euler method; where I is the 
identity matrix, r is a positive real number, and a > is the step size. 

2) A more general autonomous linear system descibed by x(/c + 1) = —pi + Wx(k), where 
W is any real symmetric matrix whose diagonal elements are zero, and I denotes the 
identity matrix and p is a positive real number. 

Our analysis shows that: 

1) The "SIR" of any state converges to a constant value, called "Ultimate SIR", in a finite 
time in the above-mentioned discrete-time linear systems. 

2) The "Ultimate SIR" in the first system above is equal to -p 2 — where \ max is the maximum 
(positive) eigenvalue of the matrix W. These results are in line with those of [9] where 
corresponding continuous-time linear system is examined. 

3) The "Ultimate SIR" in the second system above is equal to where A m is the eigenvalue 
of W which satisfy |A m — p\ = max{|Aj — p\}fLi if p is accordingly determined from the 
interval < p < 1 as described in (57). 

In the later part of the paper, we use the introduced "Ultimate SIR" to stabilize the (originally 
unstable) networks. It's shown that the proposed Discrete-Time "Stabilized"-Autonomous-Linear- 
Networks-with-Ultimate-SIR" exhibit features which are generally attributed to Discrete-Time 
Hopfield Networks. Taking the sign of the converged states, the proposed networks are applied to 
binary associative memory design. Computer simulations show the effectiveness of the proposed 
networks as compared to traditional discrete Hopfield Networks. 

As far as design of the design of the weight matrices are concerned, we also present an outer- 
products (Hebbian-learning)-based method, and show that if the prototype vectors are orthogonal 
in the proposed DSAL-U"SIR"1 network, then the ultimate SIR 9 ult is equal to where iV 
is the dimension of the network and L is the prototype vectors. 

Appendix I 

Proof of Proposition 5: 

The solution of the proposed network DSALU-"SIR"1 in eq.(8) is 
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x(Jfe) = (i + a(-rl + W)) fc x(0) (1 10) 
Let's denote the system matrix as 

M = I + a(-rl + W) (111) 

From eq.(100) and (102), 

W = Q-LI (112) 
where L is the number of orthogonal prototype vector. Using eq.(112) in (111) gives 

M= (l-a(r + L))l + aQ (113) 

and since d s e (-1,+!)^, 

Q 2 = iVQ (114) 

where N is the dimension of the system, i.e., the number of states, and Q is given in (100). 
Next, we examine the powers of matrix M) since the solution of the system is x(fc) = M fc x(0): 
First let's define b and c as follows 

b = l-a(r + L) (115) 

c = 1 -a(r + L-N) (116) 

From eq.(113) and (115), 

M = 6I + (t(1)Q (117) 

where 

a(l) = a (118) 
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The matrix M 2 is 

M 2 = b 2 I + cr(2)Q (119) 

where a(2) is equal to 

a(2) = a(b + c) (120) 
Similarly, the matrix M 2 is obtained as 

M 3 = b 3 I + cr(3)Q (121) 

where a (3) is 

a(3) = a(b 2 + be + c 2 ) (122) 

For k = 4, 

M 4 = 6 4 I + <t(4)Q (123) 

where a (4) is 

cr(4) = a{b 3 + b 2 c + be 2 + c 3 ) (124) 
So, when we continue, we observe that the k'th power of the matrix M is obtained as 

M k = b k I + a{k)Q (125) 

where a(k) is 

k 

a(k) = aJ2 b k - m c m - 1 (126) 

m=l 

where b = 1 — a(r+L) and c = 1 — a(r + L — N) as defined in eq.(115) and (116), respectively. 
Let's define the following constant ip 

b 1 - q(r + L) 
V= c= l-a(r + L-N) (127) 
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Using (127) in (126) results in 



k 

<j(k) = ac^ 1 ]T V k ~ m ( 128 ) 

m=l 



Summing —cr{k) with ipa(k) yields 



k i 

a(k) = ^—^ac k - 1 (129) 
ip - 1 



From eq.(125) and (129), the matrix M fc is written as follows 

k i 

M k = b k I + ac k ' l if - Q (130) 
Using the definition of b and c in eq.(115) and (116), respectively, in (130) gives 



M k = M tp (k) + M sp (k) (131) 

where 



M tp (k) = (l - a(r + L)) k l - -L(l - «( r + L))*Q (132) 

and 

fl - air + L - N)) k 
M sp (k) = ± — (133) 

In above equations, the number of network dimension (N) is much larger than the number 
of prototype vector (L), i.e. N >> L. In Hopfield networks, theoretically, L is in the range of 
%15 of iV (e.g. [3]). So, N > r + L by choosing r accordingly. The learning factor positive 
a is also typically a relatively small number less than 1. Therefore, (l — a(r + L)^j < 1 and 
(l - a(r + L - iV)) > 1. This means that 1) the NL tp (k) in eq.(132) vanishes (aproaches to 
zero) within a finite step number and 2) what shapes the steady state behavior of the system 
is merely M_ sp . 

Mtp(fc) ^0, k > k T (134) 
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We call the matrices M. tp (k) and M sp (k) in (132) and (133) as transitory phase part and 
steady phase part, respectively, of the matrix M fc . 

So, the steady phase solution is obtained from eq.(ll), (131) and (133) 



x sp (A;) = M sp (A;)x(0) (135) 
(l -a(r + L-N)^ 



aN 

Let's define the interference vector, 3 sp (k) as 



-Qx(0), k > k T (136) 



J sp (k) = W Xsp (fc) (137) 

From eq.(112), (114) and (137) 

J sp (k) = (Q - LT)x sp (k) (138) 

(l - a(r + L - N)) k 
= A ^ ^(TV-L)Qx(O) (139) 

So, dividing vector x. sp (k) of eq.(136) to J sp (k) of eq.(138) elementwise and comparing the 
outcome with the "SIR" definition in eq.(3) results in 

W = iV^I' i = 1 >--->w ( 14 °) 

which completes the proof. 
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Fig. 1. The figure shows the ratio K n in eq.(49) for some different AA values (A = 5, £ = 0.11). 
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Fig. 2. The figure shows same examples of ratio n n in eq.(90) for some different AA values (A = 5). 
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Percentage of correctly recovered prototype vectors, N=8 




Discrete Hopfield Network DSALU-"SIR"1 DSALU-"SIR"2 



Fig. 3. The figure shows percentage of correctly recovered desired patterns for all possible initial conditions in example 1 for 
the proposed DSALU-"SIR" and Sign"SIR"NN as compared to traditional Hopfield network with 8 neurons. 
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Percentage of correctly recovered prototype vectors, N=16 




Discrete Hopfield Network DSALU-"SIR"1 DSALU-"SIR"2 



Fig. 4. The figure shows percentage of correctly recovered desired patterns for all possible initial conditions in example 2 for 
the proposed DSALU-"SIR" and Sign"SIR"NN as compared to traditional Hopfield network with 16 neurons. 
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